Sketching Structured Matrices for Faster Nonlinear Regression
نویسندگان
چکیده
Motivated by the desire to extend fast randomized techniques to nonlinear lp regression, we consider a class of structured regression problems. These problems involve Vandermonde matrices which arise naturally in various statistical modeling settings, including classical polynomial fitting problems, additive models and approximations to recently developed randomized techniques for scalable kernel methods. We show that this structure can be exploited to further accelerate the solution of the regression problem, achieving running times that are faster than “input sparsity”. We present empirical results confirming both the practical value of our modeling framework, as well as speedup benefits of randomized regression.
منابع مشابه
Isometric sketching of any set via the Restricted Isometry Property
In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices. This class includes several matrices for which matrix-vector multiply can be computed in log-linear time, providing efficient dimensionality reduction of general sets. In particular, we show that using such matrices any set from high d...
متن کاملComputational method based on triangular operational matrices for solving nonlinear stochastic differential equations
In this article, a new numerical method based on triangular functions for solving nonlinear stochastic differential equations is presented. For this, the stochastic operational matrix of triangular functions for It^{o} integral are determined. Computation of presented method is very simple and attractive. In addition, convergence analysis and numerical examples that illustrate accuracy and eff...
متن کاملNonlinear inelastic static analysis of plane frames with numerically generated tangent stiffness matrices
For the nonlinear analysis of structures using the well known Newton-Raphson Method, the tangent stiffness matrices of the elements must be constructed in each iteration. Due to the high expense required to find the exact tangent stiffness matrices, researchers have developed novel innovations into the Newton-Raphson method to reduce the cost and time required by the analysis. In this paper, a ...
متن کاملBinary embeddings with structured hashed projections
We consider the hashing mechanism for constructing binary embeddings, that involves pseudo-random projections followed by nonlinear (sign function) mappings. The pseudorandom projection is described by a matrix, where not all entries are independent random variables but instead a fixed “budget of randomness” is distributed across the matrix. Such matrices can be efficiently stored in sub-quadra...
متن کاملFast nonlinear embeddings via structured matrices
We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above that, the presented framework covers multivariate randomized functions. As a byproduct, we propose an algorithmic approach that also leads to a significant re...
متن کامل